MOMDP-based target search mission taking into account the human operator's cognitive state

This study discusses the application of sequential decision making under uncertainty and mixed observability in a mixed-initiative robotic target search application. In such a robotic mission, two agents, a ground robot and a human operator, must collaborate to reach a common goal using, each in turn, their recognized skills. The originality of the work relies in considering that the human operator is not a providential agent when the robot fails. Using the data from previous experiments, a Mixed Observability Markov Decision Process (MOMDP) model was designed, which allows to consider aleatory failure events and the partial observable human operator's state while planning for a long-term horizon. Results show that the collaborative system was in general able to successfully complete or terminate the mission, even when many simultaneous sensors, devices and operators failures happened. So, the mixed-initiative framework highlighted in this study shows the relevancy of taking into account the cognitive state of the operator, which permits to compute a policy for the sequential decision problem which prevents to re-planning when unexpected (but known) events occurs

A Game Theoretical Formulation of a Decentralized Cooperative Multi-Agent Surveillance Mission

This paper presents a multi-aerial-robot coordination game theoretical approach to perform a surveillance mission in a well-structured environment. Such a mission consists in constantly visiting a set of points of interest while minimizing the time interval between successive visits (idleness). The proposed approach optimizes the agents' action selection based on an N-player (cooperative) game framework. The main contributions are: (i) the formulation of an original player's utility function composed of parameters that are independent from the action choices of the others players; (ii) the demonstration that the game solution is the Nash equilibrium, and this equilibrium can be obtained by optimizing separately/individually the single player's action choice; (iii) the proposal of a decentralized algorithm used to conduct the mission, which works considering minimum communication among players. Simulations evaluate the different policies obtained, which are compared using as metric the average idleness of all points of interest. The proposed framework allows for the decrease of the idleness of watched points compared to random action selection, while keeping some kind of randomness of motion (measured by a predictability metric), which can likely be desired to curb the prediction of the team surveillance strategy by an intruder

Towards human-robot interaction: a framing effect experiment

Decision making is a critical issue for humans operating unmanned vehicles. However, it is well admitted that many cognitive biases affect human judgments, leading to suboptimal or irrational decisions. The framing effect is a typical cognitive bias causing people to react differently depending on the context,the probability of the outcomes and how the problem is presented (loss vs. gain). There is a need to better understand the effects of these biases in operational contexts to optimize human-robot interactions. We therefore conducted an experiment involving a framing paradigm in a search and rescue mission (earthquake) and in a Mars rock sampling mission. We manipulated the framing (positive vs. negative) and the probability of the outcomes. Our findings revealed that the way the problem was presented (positively or negatively framed) and the emotional commitment (saving lives vs. collecting the good rock) statistically affected the choices made by the human operators

Predicting Human Operator’s Decisions Based on Prospect Theory

The aim of this work is to predict human operator's decisions in a specific operational context, such as a cooperative human-robots mission, by approximating her utility function based on Prospect Theory. To this aim, a within-subject experiment was designed in which the human operator has to decide with limited time and incomplete information. This experiment also involved a framing effect paradigm, a typical cognitive bias causing people to react differently depending on the context. Such an experiment allowed to acquire data concerning the human operator's decisions in two different mission scenarios: search and rescue and Mars rock sampling. The framing was manipulated (e.g. positive vs. negative) and the probability of the outcomes causing people to react differently depending on the context. Statistical results observed for this experiment supported the hypothesis that the way the problem was presented (positively or negatively framed) and the emotional commitment affected the human operator's decisions. Thus, based on the collected data, the present work is willed to propose: (i) a formal approximation of the human operator's utility function founded on the Prospect Theory; and (ii) a model used to predict the human operator's decisions based on the economics approach of multi-dimensional consumption bundle and Prospect Theory. The obtained results, in terms of utility function fit and prediction accuracy, are promising and show that similar modeling and prediction method should be taken into account when an intelligent cybernetic system drives human-robots interaction. The advantage of predicting the human operator's decision, in this operational context, is to anticipate her decision, given the way a question is framed to the human operator. Such a predictor lays the foundation for the development of a decision-making system capable of choosing how to present the information to the operator while expecting to align her decision with the given operational guideline

Towards mixed-initiative human-robot interaction : a cooperative human-drone team framework

L’interaction homme-robot est un domaine qui en est encore à ses balbutiements.Les développements se sont avant tout concentrés sur l’autonomie et l’intelligence artificielle et doter les robots de capacités avancées pour exécuter des tâches complexes. Dans un proche avenir, les robots développeront probablement la capacité de s’adapter et d’apprendre de leur environnement. Les robots ont confiance, ne s’ennuient pas et peuvent fonctionner dans des environnements hostiles et dynamiques - tous des attributs souhaités à l’exploration spatiale et aux situations d’urgence ou militaires. Ils réduisent également les coûts de mission, augmentent la flexibilité de conception et maximisent la production de données. Cependant, lorsqu’ils sont confrontés à de nouveaux scénarios et à des événements inattendus, les robots sont moins performants par rapport aux êtres humains intuitifs et créatifs (mais aussi faillibles et biaisés). L’avenir exigera que les concepteurs de mission équilibrent intelligemment la souplesse et l’ingéniosité des humains avec des systèmes robotiques robustes et sophistiqués. Ce travail de recherche propose un cadre formel, basé sur la théorie de jeux, pour une équipe de drones qui doit coordonner leurs actions entre eux et fournir à l’opérateur humain des données suffisantes pour prendre des décisions « difficiles » qui maximisent l’efficacité de la mission, selon certaines directives opérationnelles. Notre première contribution a consisté à présenter un cadre décentralisé et une fonction d’utilité pour une mission de patrouille avec une équipe de drones. Ensuite, nous avons considéré l’effet de cadrage, ou « framing effect » en anglais, dans le contexte de notre étude,afin de mieux comprendre et modéliser à terme certains processus décisionnels sous incertitude.Ainsi, nous avons réalisé deux expérimentations avec 20 et 12 participants respectivement. Nos résultats ont révélé que la façon dont le problème a été présenté (effet de cadrage positif ou négatif), l’engagement émotionnel et les couleurs du texte ont affecté statistiquement les choix des opérateurs humains. Les données expérimentales nous ont permis de développer un modèle d’utilité pour l’opérateur humain que nous cherchons à intégrer dans la boucle décisionnelle du système homme-robots. Enfin, nous formalisons et évaluons l’ensemble du cadre proposé où nous "fermons la boucle" à travers une expérimentation en ligne avec 101 participants. Nos résultats suggèrent que notre approche permet d’optimiser le système homme-robots dans un contexte où des décisions doivent être prises dans un environnement incertain.Human-robot interaction is a field that is still in its infancy. Developments havefocused on autonomy and artificial intelligence, and provide robots with advanced capabilitiesto perform complex tasks. In the near future, robots will likely develop the ability to adapt andlearn from their surroundings. Robots have reliance, do not get bored and can operate in hostileand dynamics environments - all attributes well suited for space exploration, and emergency ormilitary situations. They also reduce mission costs, increase design flexibility, and maximizedata production. However, when coped with new scenarios and unexpected events, robots palein comparison with intuitive and creative human beings. The future will require that missiondesigners balance intelligently the flexibility and ingenuity of humans with robust and sophisticatedrobotic systems. This research work proposes a game-theoretic framework for a drone teamthat must coordinate their actions among them and provide the human operator sufficient datato make “hard” decisions that maximize the mission efficiency, according with some operationalguidelines. Our first contribution was to present a decentralized framework and utility functionfor a drone-team patrolling mission. Then, we considered the framing effect in the context of ourstudy, in order to better understand and model certain human decision-making processes underuncertainty. Hence, two experiments were conducted with 20 and 12 participants respectively.Our findings revealed that the way the problem was presented (positive or negative framing), theemotional commitment and the text colors statistically affected the choices made by the humanoperators. The experimental data allowed us to develop a utility model for the human operatorthat we sought to integrate into the decision-making loop of the human-robot system. Finally,we formalized and evaluated the close-loop of the whole proposed framework with a last onlineexperiment with 101 participants. Our results suggest that our approach allow us to optimize thehuman-robot system in a context where decisions must be made in an uncertain environment

Blockchain-Based Multi-UAV Surveillance System

This paper describes a framework for a Multi-UAV surveillance system based on a blockchain allowing to coordinate the UAVs and to manage financial exchanges between the different users. The objective of the system is to allow a set of Points-Of-Interest (POI) to be surveyed by a set of autonomous UAVs which cooperate to minimize the time between successive visits while having an unpredictable behavior to prevent external agents from learning their movements. The system can be seen as a marketplace where the UAVs are the service providers and the POIs are the service seekers. It is based on a blockchain embedded on the UAVs and on some nodes on the ground which has two main functionalities. The first one is to organize the routes of each UAV through an efficient Game Theoretical decision algorithm implemented in a smart contract. The second one is to allow financial transactions between the system and its users: the POIs subscribe to surveillance services by buying tokens and the UAVs are paid in tokens for the provided services. The first implementation tests show that the IOTA blockchain could be a good blockchain candidate to be integrated in the multi-UAV surveillance system already developed in our lab